Search AI Products and News

IBM on Monday launched its latest mainframe hardware, the IBM z17. This fully encrypted mainframe, powered by the IBM Telum II processor, is designed for over 250 AI use cases, including AI agents and generative AI applications. While mainframes may be considered legacy technology by some, 71% of Fortune 500 companies still use them, according to sources. According to market research firm Market Research Future, by 2024...

Following the release of the official Tencent HunYuan T1 last week, many users have been eager to know when this new version would be available on Yuanbao. In the latest news, Tencent officially announced that the official version of HunYuan T1 and the latest version of DeepSeek V3 are now available, bringing users a brand-new experience. HunYuan T1 is Tencent's self-developed deep thinking model, and compared to the previous T1 Preview version, it has undergone comprehensive upgrades. The new version not only improves speed and performance but also significantly enhances overall effectiveness, achieving second-level response times, and...

Former Intel CEO Pat Gelsinger recently criticized Nvidia's pricing strategy for its AI GPUs on the Acquired podcast during Nvidia's 2025 GPU Technology Conference, arguing that the high cost makes them unsuitable for large-scale AI inference tasks. Gelsinger highlighted that inference is crucial for deploying AI models and that the industry should focus more on inference, a domain where Nvidia's technology falls short in terms of cost-effectiveness. Image source omitted.

Cerebras Systems recently announced the construction of six new data centers across North America and Europe to bolster its artificial intelligence (AI) inference capabilities. This expansion significantly increases the company's computing power, supporting the growth of various AI applications. The plan allocates 85% of computing capacity to the US, with three facilities already operational in Santa Clara and Stockton, California, and Dallas, Texas. Upcoming centers will be located in Minneapolis (expected Q2 2025), Oklahoma City, and Montreal.

Alibaba has open-sourced its new inference model, Tongyi Qianwen QwQ-32B, enabling users with consumer-grade GPUs to achieve surprisingly high performance. This makes advanced AI capabilities accessible to a wider audience.

DeepSeek launched an official account on Zhihu and published a technical article, "DeepSeek-V3/R1 Inference System Overview," detailing its model inference system optimization and cost-profit margin information for the first time, marking the end of the highly anticipated DeepSeek Open Source Week. The article highlights two key optimization goals for the DeepSeek-V3/R1 inference system: higher throughput and lower latency. To achieve these goals, DeepSeek employs large-scale cross-node expert parallelism (EP) technology, although...

On February 25th, Anthropic officially released its latest reasoning model - Claude 3.7 Sonnet. This model, with its innovative design and significantly improved programming capabilities, quickly became an industry focus. The core innovation of Claude 3.7 Sonnet lies in its fusion of rapid response and in-depth reasoning capabilities, simulating human thought processes to provide users with a more seamless interactive experience.

In an era of rapid development in artificial intelligence, the intelligence level of large models continues to improve, but the challenges of inference system efficiency are becoming increasingly apparent. Addressing high inference loads, reducing inference costs, and shortening response times has become an important issue faced by the industry. Kimi has collaborated with the MADSys laboratory at Tsinghua University to launch the KVCache-based Mooncake inference system design plan, which will be officially released in June 2024. The Mooncake inference system enhances efficiency through its innovative approach.

Recently, AI chip company Groq, based in California, has reached a partnership agreement with Saudi Arabia's oil giant Aramco to establish a large data center in Saudi Arabia. The aim of this data center is to enhance Saudi Arabia's capabilities in the field of artificial intelligence and further promote the diversification of the country's economy. Groq's CEO, Jonathan Ross, stated that the data center is expected to be operational by the end of the year, initially equipped with 19,000 Language Processing Units (LPUs). Image source note: Picture

This article outlines the latest developments in the AI inference chip field, particularly the results of the MLPerf Inference v4.1 competition, which gathered the latest chips from companies such as AMD, Google, UntetherAI, Cerebras, FuriosaAI, and Nvidia. Nvidia's H200 and GH200 chips maintain a lead in the 'Data Center Closed' category, but chips from other companies excelled in specific tasks, especially in terms of energy efficiency. AMD's Instinct Accelerator.

Cerebras Systems Inc. announced the launch of what it claims to be the 'world's fastest' AI inference service, challenging Nvidia Corp. The service aims to perform AI tasks at greater speed and lower cost. It is based on the WSE-3 processor, which has over 900,000 computing cores and 44GB of memory, with a core count 52 times that of Nvidia's H100. It claims an inference speed of up to 1,000 tokens per second, which is 20 times faster than Nvidia's comparable service, and offers a price-performance ratio 100 times better. The service is offered for free.

Cerebras Systems has launched Cerebras Inference, claiming it to be the fastest AI inference service in the world, achieving performance that surpasses traditional GPU-based systems by 20 times with significantly improved cost-effectiveness, particularly suitable for processing large language models (LLMs). Its 8B version processes 1800 tokens per second, while the 70B version processes 450 tokens, with speed and cost-performance far exceeding NVIDIA GPU solutions.

Meta engineers estimate that next year, there will be an addition of 1 to 2 million Nvidia H100 graphics processors for global AI inference. If all of them are used for language model generation, the energy consumption would fall within a reasonable range, generating 100,000 tokens per person per day. Only two additional nuclear power plants would be required to provide sufficient power for global AI inference. Model training faces limitations due to data scarcity, and researchers are working to improve model efficiency. In the next 3-4 years, it is expected that we will know whether current technology can achieve general AI.